To meet the demand for wireless data traffic having increased since deployment of 4G communication systems, efforts have been made to develop an improved 5G or pre-5G communication system. Therefore, the 5G or pre-5G communication system is also called a ‘Beyond 4G Network’ or a ‘Post LTE System’.
The 5G communication system is considered to be implemented in higher frequency (mmWave) bands, e.g., 60 GHz bands, so as to accomplish higher data rates. To decrease propagation loss of the radio waves and increase the transmission distance, the beamforming, massive multiple-input multiple-output (MIMO), Full Dimensional MIMO (FD-MIMO), array antenna, an analog beam forming, large scale antenna techniques are discussed in 5G communication systems.
In addition, in 5G communication systems, development for system network improvement is under way based on advanced small cells, cloud Radio Access Networks (RANs), ultra-dense networks, device-to-device (D2D) communication, wireless backhaul, moving network, cooperative communication, Coordinated Multi-Points (CoMP), reception-end interference cancellation and the like.
In the 5G system, Hybrid FSK and QAM Modulation (FQAM) and sliding window superposition coding (SWSC) as an advanced coding modulation (ACM), and filter bank multi carrier (FBMC), non-orthogonal multiple access (NOMA), and sparse code multiple access (SCMA) as an advanced access technology have been developed.
A fourth-generation mobile communication system such as 3GPP Long Term Evolution (LTE), and Worldwide Interoperablility for Microwave Access (WiMax), which is currently used, has an Orthogonal Frequency-division Multiplexing (OFDM) scheme based physical layer transmission structure. Further, the OFDM scheme satisfies the need of a user who wants a fast data transmission rate by increasing efficiency of frequency resources using orthogonality of a carrier. However, in the current OFDM scheme, a guard band is required because a large amount of leakage power between frequency bands is generated. To overcome this disadvantage, a Filter Bank Multi Carrier (FBMC) transmission technology has been studied as a physical layer transmission technique for fifth generation mobile communication. The FBMC can obtain high frequency efficiency by reducing the leakage power using a frequency filter, but its implementation is complex.
A Quadrature Amplitude Modulation-FBMC (QAM-FBMC) uses a symbol which does not have orthogonality so that Inter-Symbol Interference (ISI) occurs in the receiver and an accordingly generated amount is very small. Therefore, the amount is insignificant in comparison with a noise when a modulation scheme of a low order is used, but a problem occurs because a noise ratio of an amount of ISI increases when a modulation scheme of a high order is used.
Therefore, a method of resolving a problem of converging (bottlenecking) of the SINR value, which is a weak point of the QAM-FBMC transmission scheme, while maintaining a merit in that a QAM symbol is used and complexity of a transceiver is low.